<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
    <meta charset="utf-8" />
    <meta http-equiv="X-UA-Compatible" content="IE=edge" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
      <link rel="shortcut icon" href="../img/favicon.ico" />
    <title>kunpeng - MKLorum</title>
    <link rel="stylesheet" href="../css/theme.css" />
    <link rel="stylesheet" href="../css/theme_extra.css" />
        <link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/styles/github.min.css" />
    
      <script>
        // Current page data
        var mkdocs_page_name = "kunpeng";
        var mkdocs_page_input_path = "kunpeng.md";
        var mkdocs_page_url = null;
      </script>
    
    <!--[if lt IE 9]>
      <script src="../js/html5shiv.min.js"></script>
    <![endif]-->
      <script src="https://cdnjs.cloudflare.com/ajax/libs/highlight.js/11.8.0/highlight.min.js"></script>
      <script>hljs.highlightAll();</script> 
</head>

<body class="wy-body-for-nav" role="document">

  <div class="wy-grid-for-nav">
    <nav data-toggle="wy-nav-shift" class="wy-nav-side stickynav">
    <div class="wy-side-scroll">
      <div class="wy-side-nav-search">
          <a href=".." class="icon icon-home"> MKLorum
        </a><div role="search">
  <form id ="rtd-search-form" class="wy-form" action="../search.html" method="get">
      <input type="text" name="q" placeholder="Search docs" aria-label="Search docs" title="Type search term here" />
  </form>
</div>
      </div>

      <div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
              <ul>
                <li class="toctree-l1"><a class="reference internal" href="..">Home</a>
                </li>
              </ul>
              <ul class="current">
                <li class="toctree-l1 current"><a class="reference internal current" href="#">kunpeng</a>
    <ul class="current">
    <li class="toctree-l2"><a class="reference internal" href="#_1">目录</a>
        <ul>
    <li class="toctree-l3"><a class="reference internal" href="#ecs">一、华为云ECS验证</a>
        <ul>
    <li class="toctree-l4"><a class="reference internal" href="#1vpcecs">1.购买华为云虚拟私有云VPC和弹性云服务器ECS</a>
    </li>
    <li class="toctree-l4"><a class="reference internal" href="#2">2.环境搭建</a>
    </li>
    <li class="toctree-l4"><a class="reference internal" href="#3">3.准备测试</a>
    </li>
    <li class="toctree-l4"><a class="reference internal" href="#4">4.构建本地示例应用</a>
    </li>
        </ul>
    </li>
        </ul>
    </li>
    </ul>
                </li>
              </ul>
              <ul>
                <li class="toctree-l1"><a class="reference internal" href="../shengteng/">shengteng</a>
                </li>
              </ul>
      </div>
    </div>
    </nav>

    <section data-toggle="wy-nav-shift" class="wy-nav-content-wrap">
      <nav class="wy-nav-top" role="navigation" aria-label="Mobile navigation menu">
          <i data-toggle="wy-nav-top" class="fa fa-bars"></i>
          <a href="..">MKLorum</a>
        
      </nav>
      <div class="wy-nav-content">
        <div class="rst-content"><div role="navigation" aria-label="breadcrumbs navigation">
  <ul class="wy-breadcrumbs">
    <li><a href=".." class="icon icon-home" aria-label="Docs"></a></li>
      <li class="breadcrumb-item active">kunpeng</li>
    <li class="wy-breadcrumbs-aside">
    </li>
  </ul>
  <hr/>
</div>
          <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
            <div class="section" itemprop="articleBody">
              
                <h1 id="olmo">OLMo適配昇騰云</h1>
<p>OLMo（Open Language Model）是由Allen AI（AI2，艾伦人工智能研究所）开发的一个完全开源开放的大型语言模型（LLM）框架，设计初衷是为了通过开放研究，促进学术界和研究人员共同研究语言模型的科学。OLMo框架提供了一系列的资源，包括数据、训练代码、模型权重以及评估工具，以便研究人员能够更深入地理解和改进语言模型。
<a href="https://github.com/allenai/OLMo">OLMo仓库地址</a></p>
<h2 id="_1">目录</h2>
<h3 id="ecs">一、华为云ECS验证</h3>
<h4 id="1vpcecs">1.购买华为云虚拟私有云VPC和弹性云服务器ECS</h4>
<p>详解教程参考链接：<a href="https://blog.csdn.net/qq_54958500/article/details/143502238">创建华为云弹性云服务器ECS流程</a>
已有忽略</p>
<p><img alt="alt text" src="../picture/image.png" /></p>
<h4 id="2">2.环境搭建</h4>
<p>python版本为3.9,</p>
<pre><code class="language-python">#環境配置
git clone https://github.com/allenai/OLMo.git
cd OLMo
pip install -e .[all]
</code></pre>
<p>使用ModelScope获取模型参数
通过ModelScope下载到当前文件夹下</p>
<pre><code class="language-python">pip install modelscope
modelscope download --model LLM-Research/OLMo-7B  --local_dir OLMo-7B
</code></pre>
<h4 id="3">3.准备测试</h4>
<p>修改代碼：
OLMo-7B/olmo/util.py, line 718,</p>
<pre><code class="language-python">def save_hf_dataset_to_disk(
    dataset: datasets.DatasetDict | datasets.Dataset,
    hf_path: str,
    name: Optional[str],
    split: str,
    datasets_dir: PathOrStr,
):
</code></pre>
<p>修改為：</p>
<pre><code class="language-python">#添加下行代碼
from tpying import Union
def save_hf_dataset_to_disk(
    #修改下行代碼
    dataset: Union[datasets.DatasetDict, datasets.Dataset],
    hf_path: str,
    name: Optional[str],
    split: str,
    datasets_dir: PathOrStr,
):

新建测试文件run.py,复制下面代码，使用python run.py准备测试

~~~python
from transformers import pipeline
olmo_pipe = pipeline(&quot;text-generation&quot;, model=&quot;OLMo-7B&quot;)
print(olmo_pipe(&quot;Language modeling is&quot;))
</code></pre>
<p>生成结果（略有差异）：</p>
<p><img alt="alt text" src="../picture/cpu結果.png" /></p>
<h4 id="4">4.构建本地示例应用</h4>
<h5 id="41-ip">4.1 <a href="https://blog.csdn.net/qq_54958500/article/details/143517954">查看弹性公网IP和设置开放端口</a></h5>
<p>查看后得到用于登录Web应用界面的IP地址。这里为:http://1.94.197.211:7860/</p>
<h5 id="42-gradioweb">4.2 使用gradio构建Web应用界面</h5>
<pre><code class="language-python">pip install gradio
</code></pre>
<h5 id="43-gradio_runpy">4.3 新建gradio_run.py，代码如下</h5>
<pre><code class="language-python">from transformers import AutoTokenizer, AutoModelForCausalLM
import torch
import gradio as gr
device = &quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;
tokenizer = AutoTokenizer.from_pretrained(&quot;OLMo-7B&quot;, trust_remote_code=True)  
olmo = AutoModelForCausalLM.from_pretrained(
    &quot;OLMo-7B&quot;,
    low_cpu_mem_usage=True,
    trust_remote_code=True
).to(device).eval()

def olmo_inference(text):
    inputs = tokenizer([text], return_tensors='pt', return_token_type_ids=False)
    with torch.no_grad():
        response = olmo.generate(**inputs, max_new_tokens=250, do_sample=True, top_k=50, top_p=0.95)
        return tokenizer.batch_decode(response, skip_special_tokens=True)[0]
    return &quot;error&quot;

# 创建 Gradio 接口
iface = gr.Interface(
    fn=olmo_inference,
    inputs=&quot;text&quot;,
    outputs=&quot;text&quot;,
    title=&quot;调用olmo模型&quot;,
    description=&quot;输入文本，模型调用進行文本補全。&quot;
)

# 启动服务
iface.launch(server_name=&quot;0.0.0.0&quot;, server_port=7860)


</code></pre>
<p>nohup 运行gradio_run.py，并在浏览器转到http://1.94.197.211:7860/打开服务页面，可以输入以下文本让模型推理（注意：推理时间较长，请耐心等待）</p>
<pre><code class="language-text">Language modeling is
</code></pre>
<p>运行结果：
<img alt="alt text" src="./picture/image-1.png" /></p>
              
            </div>
          </div><footer>
    <div class="rst-footer-buttons" role="navigation" aria-label="Footer Navigation">
        <a href=".." class="btn btn-neutral float-left" title="Home"><span class="icon icon-circle-arrow-left"></span> Previous</a>
        <a href="../shengteng/" class="btn btn-neutral float-right" title="shengteng">Next <span class="icon icon-circle-arrow-right"></span></a>
    </div>

  <hr/>

  <div role="contentinfo">
    <!-- Copyright etc -->
  </div>

  Built with <a href="https://www.mkdocs.org/">MkDocs</a> using a <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a> provided by <a href="https://readthedocs.org">Read the Docs</a>.
</footer>
          
        </div>
      </div>

    </section>

  </div>

  <div class="rst-versions" role="note" aria-label="Versions">
  <span class="rst-current-version" data-toggle="rst-current-version">
    
    
      <span><a href=".." style="color: #fcfcfc">&laquo; Previous</a></span>
    
    
      <span><a href="../shengteng/" style="color: #fcfcfc">Next &raquo;</a></span>
    
  </span>
</div>
    <script src="../js/jquery-3.6.0.min.js"></script>
    <script>var base_url = "..";</script>
    <script src="../js/theme_extra.js"></script>
    <script src="../js/theme.js"></script>
      <script src="../search/main.js"></script>
    <script>
        jQuery(function () {
            SphinxRtdTheme.Navigation.enable(true);
        });
    </script>

</body>
</html>
